03a-TIMCI-SPA-fa Audit Analysis

Lucas Silbernagel

23 September, 2021

library(dplyr)
library(readr)
library(ggplot2)
library(openxlsx)
library(knitr)
library(tibble)
library(stringr)
library(stringi)
library(readxl)
library(lubridate)
library(shiny)
library(plotly)

1 Loading the Data and Removal of Training Data

# Unzip and extract ODK data from ODK zip
df <- as.data.frame(extract_data_from_odk_zip(params$file_path_zip, params$file_name_csv))

# Formatting dates from integer (in ms) to time stamp
df$start <- format_date_ms(df$start)
df$end <- format_date_ms(df$end)

# filtering for events that occurred after 18th July 21
df <- subset(df, as.Date(start) > as.Date("18.07.2021", "%d.%m.%Y"))

2 Deriving New Features

2.1 Time Spent per Event

# subtracting end from start date
df$time_spent = round(as.numeric(df$end - df$start))

2.2 Question

# splitting the node strings so that only the question name remains 
df$question = sapply(df$node, create_question)

2.3 Question Decoded

df <- decode_question(df, df$question, params$codebook)

2.4 Categorical Answers Decoded

df <- decode_categories(df, params$codebook)

2.5 Time until a Response was Changed + Stream of Answer Changes

df <- df %>%
# bringing the data in the right order   
  arrange(`instance ID`, node, start) %>%
# adding two empty columns to store the new features in
  add_column(time_till_change=NA) %>%
  add_column(changed_from=NA)

# iterating over the df and computing the time it took until an answer was changed + adding what the question was before 
for (i in 1:nrow(df)){
  if (df$`old-value`[i]==df$`new-value`[i-1] && !is.na(df$`old-value`[i]) && !is.na(df$`new-value`[i-1]) ){
    df$time_till_change[i] <- round(as.numeric(df$start[i]-df$end[i-1]))
  } else{
    next
  }
}

2.6 Preview and Summary of the Final Data

head(df)
instance ID event node start end latitude longitude accuracy old-value new-value time_spent question question_decoded new_value_decoded old_value_decoded time_till_change changed_from
uuid:277e3be0-ea5a-44ec-b869-4e0c56b13c8d group questions /data/b2 2021-08-10 08:23:55 2021-08-10 12:52:13 NA NA NA NA NA 16098 b2 NA NA NA NA NA
uuid:277e3be0-ea5a-44ec-b869-4e0c56b13c8d question /data/b2/b2_3 2021-08-10 08:23:55 2021-08-10 12:52:13 NA NA NA NA 2, 3 16098 b2_3 In addition to outpatient services, what other services are provided in the facility? Daytime observation capacity, Emergency department NA NA NA
uuid:277e3be0-ea5a-44ec-b869-4e0c56b13c8d question /data/b2/b4_1 2021-08-10 08:23:55 2021-08-10 12:52:13 NA NA NA NA 1, 3, 5, 2, 4 16098 b4_1 Opening days Monday, Wednesday, Friday, Tuesday, Thursday NA NA NA
uuid:277e3be0-ea5a-44ec-b869-4e0c56b13c8d question /data/b2/b4_2 2021-08-10 08:23:55 2021-08-10 12:52:13 NA NA NA NA 2 16098 b4_2 Daily opening hours 5 to 8 hours NA NA NA
uuid:277e3be0-ea5a-44ec-b869-4e0c56b13c8d question /data/b2/b4_3 2021-08-10 08:23:55 2021-08-10 12:52:13 NA NA NA NA 0 16098 b4_3 At any time during the last 7 days has the facility been unexpectedly closed? No NA NA NA
uuid:277e3be0-ea5a-44ec-b869-4e0c56b13c8d question /data/b2/d1_a_1 2021-08-10 08:23:55 2021-08-10 12:52:13 NA NA NA NA 1, 2, 3, 4 16098 d1_a_1 Services provided to children 2-59 months Vaccination, Growth monitoring, Nutrition, Outpatient curative services NA NA NA
summary(df)
##  instance ID           event               node               start                    
##  Length:3150        Length:3150        Length:3150        Min.   :2021-07-19 09:22:16  
##  Class :character   Class :character   Class :character   1st Qu.:2021-07-28 12:17:32  
##  Mode  :character   Mode  :character   Mode  :character   Median :2021-08-10 13:03:05  
##                                                           Mean   :2021-08-06 15:03:51  
##                                                           3rd Qu.:2021-08-17 14:08:02  
##                                                           Max.   :2021-08-30 14:57:34  
##                                                                                        
##       end                      latitude       longitude      accuracy      
##  Min.   :2021-07-19 09:22:19   Mode:logical   Mode:logical   Mode:logical  
##  1st Qu.:2021-07-28 12:17:40   NA's:3150      NA's:3150      NA's:3150     
##  Median :2021-08-10 13:07:18                                               
##  Mean   :2021-08-06 15:34:50                                               
##  3rd Qu.:2021-08-17 14:13:21                                               
##  Max.   :2021-08-30 14:57:34                                               
##  NA's   :162                                                               
##   old-value          new-value           time_spent        question        
##  Length:3150        Length:3150        Min.   :    0.0   Length:3150       
##  Class :character   Class :character   1st Qu.:  101.0   Class :character  
##  Mode  :character   Mode  :character   Median :  259.0   Mode  :character  
##                                        Mean   :  523.2                     
##                                        3rd Qu.:  520.0                     
##                                        Max.   :67044.0                     
##                                        NA's   :162                         
##  question_decoded   new_value_decoded  old_value_decoded  time_till_change 
##  Length:3150        Length:3150        Length:3150        Min.   :-18.000  
##  Class :character   Class :character   Class :character   1st Qu.:  2.000  
##  Mode  :character   Mode  :character   Mode  :character   Median :  3.000  
##                                                           Mean   :  4.137  
##                                                           3rd Qu.:  3.000  
##                                                           Max.   : 49.000  
##                                                           NA's   :2989     
##  changed_from  
##  Mode:logical  
##  NA's:3150     
##                
##                
##                
##                
## 

3 General Information about the Data

no_inst = length(unique(df$`instance ID`))
no_event =  nrow(df)
earliest_start = as.Date(min(df$start)) 
latest_end = as.Date(max(df$end[!is.na(df$end)]))

Total number of instances: 14
Total number of events/questions: 3150
Examination period: 2021-07-19 - 2021-08-30

4 Grouped by Time

4.1 Events/Questions Started by Day

df_by_day <- df %>%
  mutate(start_date = as.Date(start)) %>%
  count(start_date, name = "count")

gg1 <- ggplot(df_by_day, aes(x = start_date, y = count)) +
  geom_line() +
  geom_smooth(alpha=0.5, colour="red", method="loess", se=F) +
  labs(title = "Number of Events/Questions Started by Day with Smoothed Regression Line", y =  "Number of Questions/Events Started", x = "Satrt Date") +
  theme_light() 
gg1

4.2 Questions/Events started by Weekday and Hour of the Day

df_wday_hour <- df %>%
  mutate(wday=wday(start, label=T, week_start = 1), hour=hour(start)) %>%
  count(wday, hour, name="count_wday_hour") %>%
  arrange(desc(wday))

theme_heatmap <- theme_light() +                 
  theme(panel.grid = element_blank(),            
        panel.border = element_blank(),          
        plot.title = element_text(face = "bold", size = 11, hjust = 0.5), 
        axis.ticks = element_blank(),            
        axis.title.x = element_blank(),        
        axis.title.y = element_text(size=10),   
        axis.text.y = element_text(size = 8),    
        axis.text.x = element_text(size = 10),   
        legend.position = "none")                

gg2 <- ggplot(df_wday_hour, aes(x = wday, y = hour, fill = count_wday_hour)) +
  geom_tile(colour="white") +  
  scale_fill_gradient(low = "#fff0f0", high="#940606") +  
  scale_y_reverse(breaks=c(23:0), labels=c(23:0), expand = c(0,0)) +               
  scale_x_discrete(expand = c(0,0), position = "top") +
  labs(title = "Number of Started Events/Questions by Day of Week / Hour of Day", y = "Hour of Day") +
  geom_text(aes(label = count_wday_hour), size = 2) +
  theme_heatmap  
gg2

4.3 Distribution of Time Spent per Event/Question with largest 5 % removed

df_clean = subset(df, time_spent<quantile(df$time_spent,0.95, na.rm=TRUE))

hist(df_clean$time_spent[!is.na(df_clean$time_spent)]/60, breaks=20, xlab = "Time Spent in Minutes", main = "Histogram of the Time Spent by Question")

5 Aggregated by Event/Question

5.1 Median Time Spent by Question

df_median_time_per_question <- df %>%
  filter(event=="question") %>%
  group_by(question_decoded) %>%
  summarise(median_time_spent = median(time_spent)) %>%
  arrange(desc(median_time_spent)) %>%
  mutate(median_time_spent = round(seconds_to_period(median_time_spent)))

df_median_time_per_question
question_decoded median_time_spent
How are areas for quality improvement in relation to clinical service delivery identified / agreed? 29M 22S
How are possible solutions to identified areas for quality improvement discussed? 29M 22S
How is implementation of solutions to identified areas for quality improvement supervised? 29M 22S
How is implementation of solutions to identified quality improvement needs monitored? 24M 42S
Toilet functional 21M 29S
What happens when the internet monthly data limit is reached? 21M 29S
What did the supervisor observe during the last visit? 21M 4S
Who provided the external supervision? 20M 4S
What quality improvement goals or initiatives in relation to clinical service delivery are currently ongoing in this facility? 19M 36S
Did the supervision visit result in the following outputs? 19M 16S
How many external supervision visits have occurred during the 12 last months? 19M 16S
How many healthcare providers receive supportive supervision? 19M 16S
When was the last time this facility received an external supervision visit? 19M 16S
Was IMCI part of the supervision visit? 17M 46S
What did the supervisor do during the last visit? 17M 46S
Is the emergency department separate from the outpatient area? 17M 28S
Triage for children 0-59 months 17M 28S
How do external supervisors monitor the quality of services at this facility? 16M 15S
What did the supervisor in terms of training / support during the last visit? 14M 52S
What did the supervisor review during the last visit? 14M 52S
For this health facility, who decides which services are available/on offer to clients? 13M 23S
Days per week facility for which there is normally at least one eligible healthcare professional is available to consult children 0-59 months? 13M 22S
Does someone else consult if the eligible provider is unavailable? 13M 22S
Number of days for which there was no eligible healthcare provider available to consult children 0-59 months in the last 7 days? 13M 22S
Total number of consultation rooms available for simultaneous consultation of children 13M 22S
Did the last supervision visit result in the planning of the following activities? 13M 12S
Is there a system at the facility to keep track of staff trainings and certifications? 13M 10S
When was the last time the facility committee happened? 12M 56S
Who is responsible for the day to day management of the facility? 12M 56S
How frequently is data quality checked? 12M 42S
For this health facility, who participates in the budget planning and budget development process? 11M 32S
Source of supplies / consumables / equipment for this facility? 11M 32S
What is the procedure if a caregiver is unable to pay for any of the fees associated with healthcare provided to a sick child 0-59 months in this facility? 11M 32S
Where activities were identified, were they followed upon as part of supervision? 11M 16S
Does this facility receive any external supervision? 11M 6S
Does this health facility have any current goals or initiatives for quality improvement in relation to clinical service delivery? 11M 6S
Frequency of facility committees 11M 6S
Are fees charged for the following services for healthcare provided to a sick child 0-59 months ? 10M 8S
Do the funds or resources available to the facility depend on the facility’s past or current performance against specific metrics or goals (either overall or for specific designated targets or goals)? 10M 8S
For this health facility, is the budget based on categories of types of expenses (such as staffing, medicines, equipment etc.) or based on categories of the types of services (eg. HIV services, MCH services etc.), or both? 10M 8S
For this health facility, who makes decisions about the overall staffing structure and staffing resources (how many providers and staff are hired to work here, the types of staffs are hired to work here etc.)? 10M 8S
Is there any mechanism to ensure the accuracy of recorded information on registries?
OBTAIN documents related to data quality checks (reports, feedback, checklists). 10M 3S
For what population is the facility policy to charge no fees? 9M 60S
For what kind of trainings are these mechanisms in place? 9M 51S
Training log management system 9M 51S
Does this facility have budget earmarked for staff specifically hired for and designated to provide services for infants and/or children exclusively (and not adults)? 9M 42S
For this facility, is there a budget or allocation of funds specifically designated to be used for services, staffing, medicines or equipment for infant and or child health curative services (and not other services such as PMTCT, HIV care and treatment or adult services)? 9M 42S
Source of revenue or funding of this facility? 9M 42S
How much time is taken from the facility staff to report facility data related to outpatient children 0-59 months per week? 9M 17S
Individual clinical notes 9M 17S
New visits reported to HMIS? 9M 17S
Referrals reported to HMIS? 9M 17S
Registers / ledger recording available at the facility 9M 17S
Diagnoses reported to HMIS? 8M 54S
Number of sick children per month reported to HMIS 8M 54S
Total number of children under 5 consultations reported to HMIS? 8M 54S
Total number of subgroup of # young infant/neonatal consultations reported to HMIS 8M 54S
Vaccination / monitoring area / room separate from curative services 8M 46S
Is there an “observation area” where children and young infants can stay for longer observation without admission? 8M 35S
On average, hours per day facility at least one eligible healthcare professional is available to consult children 0-59 months? 8M 35S
Communicable disease services 8M 34S
Deliveries at the facility (sub-categories if C-section vs other available) reported to HMIS? 8M 32S
Follow-up visits reported to HMIS? 8M 32S
At any time during the last 7 days has the facility been unexpectedly closed? 8M 30S
Daily opening hours 8M 30S
In addition to outpatient services, what other services are provided in the facility? 8M 30S
Opening days 8M 30S
Recommendations on alternative transports 8M 30S
Surgical services (other than C-Section) 8M 30S
Who organises transport to referral facility for children 0-59 months? 8M 30S
Services provided to children 2-59 months 8M 26S
Among registers available at the facility, which one(s) are used for HMIS outpatient children 0-59 months reporting? 8M 22S
Segregation of waste 7M 46S
Check completeness (e.g. blank counts) of (provisional or final) diagnoses over the last 20 entries 7M 44S
Check readability of (provisional or final) diagnoses over the last 20 entries 7M 44S
Check relevance of (provisional or final) diagnoses over the last 20 entries
(e.g. check diagnoses only, not symptoms are reported as diagnoses) 7M 44S
Laboratory register / ledger recording specific for children 0-59 months 7M 44S
Pharmacy records management system? 7M 44S
Pharmacy register / ledger recording specific for children 0-59 months 7M 44S
Clinical observations recorded in the registration or consultation ledger 7M 33S
Laboratory records management system? 7M 33S
Are tablets provided by the government at the facility? 7M 26S
Available modes of communication 7M 26S
Hand-washing soap / liquid soap 7M 26S
Main electricity source 7M 26S
Most commonly used source of water for the facility at this time? 7M 26S
running_water 7M 26S
Toilet in the facility for outpatient use 7M 26S
Diagnoses recorded in the consultation ledger 7M 22S
Alcohol based handrub 7M 5S
Are computers provided by the government at the facility? 7M 5S
Personal protective equipment 7M 5S
Days of the week for which there are dedicated vaccination / growth monitoring services 6M 57S
Please specify (d1.2) Back up electricity source(s)) 6M 4S
Salbutamol 6M 4S
Stock outs in the last 7 days 6M 4S
Among those services, which ones are also provided to children 0-2 months? 6M 3S
Artesunate (rectal) 6M 1S
Cloxacillin 6M 1S
Co-trimoxazole syrup / suspension 6M 1S
Gentian violet 6M 1S
IV fluid - glucose 6M 1S
IV fluid - ringer’s lactate / sodium chloride 6M 1S
Mebendazole Tablet 6M 1S
Neonatal IV needles 6M 1S
Nitrofurantoin 6M 1S
Oral artemisinin combination therapy (paediatric dose) 6M 1S
Paediatric IV needles 6M 1S
Paracetamol syrup/suspension 6M 1S
Primaquine 6M 1S
Rifampicin / Isoniazid / Pyrazinamide / Etambutol (RHZE) 6M 1S
Diazepam 5M 57S
NG tubes 5M 57S
Quinine (oral) 5M 57S
Artemether injection 5M 53S
Chlorhexidine 5M 53S
Tasks for which computers are used? 5M 48S
Artesunate (injection) 5M 44S
Cups/palladium for neonatal feeding 5M 44S
IV giving set 5M 44S
Metronidazole 5M 44S
Oral Rehydration Salts (ORS) sachets 5M 44S
Quinine (injection) 5M 44S
Rifampicin / Isoniazid (RH) 5M 44S
Stock stored in first-to-expire, first-out (FEFO) order (i.e. the stock that will expire first is the closest to the front) 5M 44S
Vitamin A (retinol) capsules 5M 44S
Zinc sulphate syrup or dispersible tablets 5M 44S
NA 5M 44S
Azithromycin 5M 40S
Gentamicin 5M 40S
Stock replenishment mechanism 5M 40S
Amoxicillin - dispersible tablets (250mg) 5M 28S
Amoxicillin - dispersible tablets (500mg) 5M 28S
Amoxicillin - syrup / suspension (250 mg) 5M 28S
Ampicillin 5M 28S
Benzylpenicillin 5M 28S
Ciprofloxacin 5M 28S
Co-amoxiclav (Amoxicillin + Clavulanic acid) 5M 28S
Amoxicillin - syrup / suspension (125 mg) 5M 20S
Ceftriaxone 5M 20S
Is the internet network provided by government accessible today? 4M 53S
Monthly data limit of internet provided by government at the facility 4M 53S
Number of computers seen in use during the assessment 4M 53S
What is the mobile phone network signal strength today (bar level on the respondent’s phone)? 4M 53S
Where do you refer children 0-59 months who require oxygen then? 4M 52S
Frequency of IT interventions in the last 7 days 4M 51S
Internet access provided by government at the facility 4M 51S
Responsible for IT maintenance / intervention 4M 51S
Average time driving to ${reffname} 4M 46S
Are there existing mechanisms for recording staff training? 4M 37S
Mechanisms used to directly inform the referral facility of the referral of a child 0-59 months? 4M 25S
Tests routinely perfomed 4M 12S
Electricity availability in past 7 days (from main or backup source) during facility opening hours 4M 2S
Number of computers available at the facility 4M 2S
Number of computers seen during the assessment 4M 2S
Tasks for which tablets are used? 4M 2S
Growth charts available 3M 57S
Guidelines for Infection Prevention and Control 3M 57S
Height board available 3M 57S
IMCI guidelines 3M 57S
Malaria testing available 3M 56S
Checklists and / or job aids for IMCI 3M 54S
Child weighing scale (250g gradation) available 3M 54S
Emergency Triage Assessment and Treatment (ETAT) guidelines 3M 54S
Guidelines for Basic Emergency Obstetric and Newborn Care (BEmONC) 3M 54S
Guidelines for Comprehensive Emergency Obstetric and Newborn Care (CEmONC) 3M 54S
IMCI guidelines on care of sick young infant 0-2 months 3M 54S
Infant weighing scale (100g gradation) available 3M 54S
Measuring tape available 3M 54S
Oxygen therapy guidelines / job aids 3M 54S
Pulse oximetry guidelines 3M 54S
Stadiometre available 3M 54S
Thermometer available 3M 54S
Consultation records management system? 3M 42S
Registration records management system? 3M 42S
By what mechanisms are you informed by the referral facility? 3M 30S
Information received back on outgoing referrals from referral facility? 3M 30S
Please specify (Mechanisms used to directly inform the referral facility of the referral of a child 0-59 months?) 3M 30S
Do you refer children who require oxygen somewhere else? 3M 29S
Do you refer young infants and neonates somewhere else? 3M 29S
Does the facility centrally record information about outgoing referrals? 3M 29S
Mechanisms used to inform caregiver about the referral of a child 0-59 months 3M 29S
Where do you in priority send children 0-59 months with severe illness who need to be referred? 3M 29S
When was the last time a data quality check was undertaken? 3M 5S
Mechanisms used to inform caregivers about recommended follow-up of children 0-59 months 2M 57S
Diagnostic testing at the facility (including any rapid diagnostic testing) 2M 40S
What information does the facility centrally record during follow-up visits? 2M 40S
Ultrasound on-site available for children 0-59 months 2M 24S
X-ray on-site available for children 0-59 months 2M 24S
At any time during the past 7 days has Haemocue been unavailable (for any reason)? 2M 20S
At any time during the past 7 days has malaria blood film microscopy been unavailable (for any reason)? 2M 20S
At any time during the past 7 days has malaria RDT been unavailable (for any reason)? 2M 20S
Haemoglobin testing / estimation 2M 20S
location 2M 8S
HIV testing 2M 3S
Pocketbook for children 2M 0S
Toilet accessible (unlocked or key available) 1M 58S
Type of toilet 1M 58S
At any time during the past 7 days has blood glucose been unavailable (for any reason)? 1M 54S
staff_childcare 1M 34S
staff_time 1M 34S
At any time during the past 7 days has pulse oximetry been unavailable in the outpatient department for any reason? 1M 28S
Countertop available 1M 23S
Fingertip available 1M 23S
Handheld device available 1M 23S
Paediatric probe available 1M 23S
Type of oxygen supply available 1M 23S
Young infant probe available 1M 23S
staff_training 1M 19S
Does the facility centrally record outcomes of outgoing referrals? 1M 18S
At any time during the past 7 days has stool testing been unavailable (for any reason)? 1M 15S
At any time during the past 7 days has urine testing been unavailable (for any reason)? 1M 15S
Stool analysis 1M 15S
Urine 1M 15S
Please select the current district 1M 3S
At any time during the past 7 days has HIV dried blood spot collection been unavailable (for any reason)? 57S
At any time during the past 7 days has HIV rapid testing been unavailable (for any reason)? 57S
At any time during the past 7 days has TB testing been unavailable (for any reason)? 57S
staff_type 57S
TB testing 57S
Number of departures among the clinical staff in the last 3 months 54S
Number of arrivals among the clinical staff in the last 3 months 50S
Thermometer functional 46S
cmpl_imci_training 16S
path_imci_training 16S
staff_status 10S
staff_end 5S

5.2 Count of Input Changes and Median Time until Input was Changed by Question

df_changes_per_question <- df %>%
  filter(event=="question", 
         !is.na(time_till_change)) %>%
  group_by(question_decoded) %>%
  summarise(count_input_changes=n(), 
            median_time_till_change=median(time_till_change), 
            sd_time_till_change=sd(time_till_change)) %>%
  arrange(desc(count_input_changes)) %>%
  mutate(median_time_till_change = round(seconds_to_period(median_time_till_change)),
         sd_time_till_change = round(seconds_to_period(sd_time_till_change), 1)) %>%
  filter(count_input_changes > 1)

df_changes_per_question
question_decoded count_input_changes median_time_till_change sd_time_till_change
staff_end 17 2S 13.8S
NA 4 3S 9S
Among registers available at the facility, which one(s) are used for HMIS outpatient children 0-59 months reporting? 3 3S 12.4S
Ampicillin 3 3S 10.4S
Artemether injection 3 3S 0.6S
At any time during the past 7 days has pulse oximetry been unavailable in the outpatient department for any reason? 3 1S 2.3S
Ciprofloxacin 3 3S 0.6S
Follow-up visits reported to HMIS? 3 3S 10.4S
For this health facility, who makes decisions about the overall staffing structure and staffing resources (how many providers and staff are hired to work here, the types of staffs are hired to work here etc.)? 3 3S 2S
How do external supervisors monitor the quality of services at this facility? 3 3S 11.6S
How much time is taken from the facility staff to report facility data related to outpatient children 0-59 months per week? 3 3S 4S
Individual clinical notes 3 3S 1.2S
Information received back on outgoing referrals from referral facility? 3 3S 1.2S
Mechanisms used to inform caregivers about recommended follow-up of children 0-59 months 3 3S 1.2S
Personal protective equipment 3 3S 1.2S
staff_status 3 4S 14.4S
Stock outs in the last 7 days 3 5S 8.1S
Available modes of communication 2 3S 0S
Azithromycin 2 3S 0S
Diazepam 2 3S 0S
Does the facility centrally record information about outgoing referrals? 2 3S 0S
Electricity availability in past 7 days (from main or backup source) during facility opening hours 2 3S 0S
For this health facility, who participates in the budget planning and budget development process? 2 4S 1.4S
For what population is the facility policy to charge no fees? 2 2S 1.4S
Gentamicin 2 3S 0S
Guidelines for Basic Emergency Obstetric and Newborn Care (BEmONC) 2 3S 2.8S
Guidelines for Comprehensive Emergency Obstetric and Newborn Care (CEmONC) 2 2S 1.4S
Is there a system at the facility to keep track of staff trainings and certifications? 2 4S 1.4S
Is there any mechanism to ensure the accuracy of recorded information on registries?
OBTAIN documents related to data quality checks (reports, feedback, checklists). 2 2S 1.4S
Mechanisms used to directly inform the referral facility of the referral of a child 0-59 months? 2 3S 0S
NG tubes 2 3S 0S
Oxygen therapy guidelines / job aids 2 3S 2.8S
Pulse oximetry guidelines 2 3S 2.8S
Quinine (injection) 2 2S 1.4S
Quinine (oral) 2 3S 0S
Registers / ledger recording available at the facility 2 2S 0.7S
Salbutamol 2 10S 11.3S
Source of supplies / consumables / equipment for this facility? 2 4S 1.4S
Stock replenishment mechanism 2 12S 13.4S
What is the procedure if a caregiver is unable to pay for any of the fees associated with healthcare provided to a sick child 0-59 months in this facility? 2 4S 1.4S

5.3 Count of Old-New Value Pairs

df_stream <- df %>%
  filter(!is.na(time_till_change)) %>%
  count(question_decoded, 
        old_value_decoded, 
        new_value_decoded, 
        name="count_value_pairs", 
        sort=TRUE) %>%
  filter(count_value_pairs > 1)

df_stream
question_decoded old_value_decoded new_value_decoded count_value_pairs
staff_end No Yes 11
staff_end Yes No 6
Electricity availability in past 7 days (from main or backup source) during facility opening hours Always available (no interruptions) Often available (interruptions <2hrs / day) 2
Follow-up visits reported to HMIS? Yes No 2
For what population is the facility policy to charge no fees? Children [0-59 months] with health insurance only All children [0-59 months] 2
Gentamicin O+ N+ 2
Guidelines for Basic Emergency Obstetric and Newborn Care (BEmONC) R N 2
Individual clinical notes Copy held by facility (informal system) Copy held by facility (formal records system), Copy held by caregiver (formal child health card) 2
Is there a system at the facility to keep track of staff trainings and certifications? Yes No 2
Oxygen therapy guidelines / job aids R N 2
staff_status Assigned Employed 2
What is the procedure if a caregiver is unable to pay for any of the fees associated with healthcare provided to a sick child 0-59 months in this facility? Service not provided, asked to come back when able to pay Other 2

6 Aggregated by Instance

6.1 Top 10 % of Duration by Instance

df_duration_per_inst <- df %>%
  group_by(`instance ID`) %>%
  summarise(duration_per_inst = max(end, na.rm=T) - min(start, na.rm=T)) %>% 
  filter(duration_per_inst>quantile(duration_per_inst, 0.9, na.rm=TRUE)) %>%
  mutate(duration_per_inst = round(seconds_to_period(duration_per_inst))) %>%
  arrange(desc(duration_per_inst))

df_duration_per_inst
instance ID duration_per_inst
uuid:f505d065-163e-42a3-8a85-25988890b584 4d 23H 44M 27S
uuid:421e5faf-9109-49da-a8f6-c2059d496391 3d 1H 51M 43S

6.2 Distribution of Duration by Instance with Top 10 % excluded

df_subsetted <- df %>%
  group_by(`instance ID`) %>%
  summarise(duration_per_inst = max(end, na.rm=T) - min(start, na.rm=T)) %>%
  filter(duration_per_inst<quantile(duration_per_inst, 0.9, na.rm=TRUE))
 
hist(as.numeric(df_subsetted$duration_per_inst/60), breaks=30, main="Duration per Instance in Minutes (outliers removed)", xlab="Duration in Minutes")

7 Irregularities and Outliers

7.1 Time Till Change Outliers (for all data without removed outliers)

df_time_till_change_outliers <- df %>% 
  filter(time_till_change>quantile(df$time_till_change, 0.9, na.rm=TRUE)) %>% 
  arrange(desc(time_till_change)) %>%
  mutate(time_till_change = round(seconds_to_period(time_till_change))) %>%
  select(`instance ID`, 
         question_decoded, 
         old_value_decoded, 
         new_value_decoded, 
         time_till_change)

df_time_till_change_outliers
instance ID question_decoded old_value_decoded new_value_decoded time_till_change
uuid:421e5faf-9109-49da-a8f6-c2059d496391 staff_end No Yes 49S
uuid:49a885da-765e-48b4-9fbd-c60b76c0c640 staff_end No Yes 35S
uuid:b4ff7422-1179-4fa6-aef5-8f3773b70854 Among registers available at the facility, which one(s) are used for HMIS outpatient children 0-59 months reporting? Consultation, Laboratory, Pharmacy Consultation, Laboratory, Pharmacy, Registration 24S
uuid:b4ff7422-1179-4fa6-aef5-8f3773b70854 Stock replenishment mechanism As required ordering by facility Periodic delivery (not as a result of ordering) 22S
uuid:2925ca93-365f-40e7-9107-0d414c60c115 Deliveries at the facility (sub-categories if C-section vs other available) reported to HMIS? Yes No 21S
uuid:2925ca93-365f-40e7-9107-0d414c60c115 Follow-up visits reported to HMIS? Yes No 21S
uuid:6f2f5d66-022c-4a0d-ad0d-ecf0e827e82f NA 1 0 21S
uuid:6f2f5d66-022c-4a0d-ad0d-ecf0e827e82f Ampicillin N+ N- 21S
uuid:49a885da-765e-48b4-9fbd-c60b76c0c640 staff_end No Yes 19S
uuid:b4ff7422-1179-4fa6-aef5-8f3773b70854 staff_end No Yes 19S
uuid:421e5faf-9109-49da-a8f6-c2059d496391 Salbutamol N+ O+ 18S
uuid:421e5faf-9109-49da-a8f6-c2059d496391 Stock outs in the last 7 days Gentamicin Co-amoxiclav 18S
uuid:49a885da-765e-48b4-9fbd-c60b76c0c640 staff_end No Yes 12S
uuid:fe61cd8c-5304-4375-98f5-3381e90e75a5 Number of arrivals among the clinical staff in the last 3 months 0 1 12S
uuid:49a885da-765e-48b4-9fbd-c60b76c0c640 staff_end No Yes 10S
uuid:6f2f5d66-022c-4a0d-ad0d-ecf0e827e82f How much time is taken from the facility staff to report facility data related to outpatient children 0-59 months per week? 48 72 10S

7.2 Histograms of Instances with Inconsistent Filling Behaviour

irregular_inst = c()
for (id in unique(df$`instance ID`)){
  bin_vec = cut(df$start[df$`instance ID`==id], 
                breaks=10, 
                labels=F)
  if (length(unique(bin_vec)) < 5) irregular_inst = c(irregular_inst, id)
}
paste0(length(irregular_inst), " out of ", length(unique(df$`instance ID`))," instances were found to have an inconsistent filling behaviour.")
## [1] "10 out of 14 instances were found to have an inconsistent filling behaviour."
last_bin_questions = c()
fig <- plot_ly(alpha=0.1)
for (id in irregular_inst){
  temp_df = df[df$`instance ID`==id,]
  temp_df$cut = cut(temp_df$start, breaks=10, labels=c("1. Part", "2. Part", "3. Part", "4. Part", "5. Part", "6. Part", "7. Part", "8. Part", "9. Part", "10. Part"))
  fig <- fig %>% add_histogram(x=temp_df$cut, name=id)
  
  last_bin_questions = c(last_bin_questions, temp_df$question_decoded[temp_df$cut=="10. Part"])
}
fig <- fig %>% layout(barmode = "overlay")
fig
kable(table(last_bin_questions) %>% as.data.frame() %>% arrange(desc(Freq)))
last_bin_questions Freq
staff_type 7
How much time is taken from the facility staff to report facility data related to outpatient children 0-59 months per week? 6
staff_end 6
Among registers available at the facility, which one(s) are used for HMIS outpatient children 0-59 months reporting? 5
Follow-up visits reported to HMIS? 5
How do external supervisors monitor the quality of services at this facility? 4
Individual clinical notes 4
New visits reported to HMIS? 4
Referrals reported to HMIS? 4
Registers / ledger recording available at the facility 4
Artemether injection 3
Ciprofloxacin 3
Deliveries at the facility (sub-categories if C-section vs other available) reported to HMIS? 3
Diagnoses reported to HMIS? 3
Did the supervision visit result in the following outputs? 3
For this health facility, who decides which services are available/on offer to clients? 3
For this health facility, who makes decisions about the overall staffing structure and staffing resources (how many providers and staff are hired to work here, the types of staffs are hired to work here etc.)? 3
For this health facility, who participates in the budget planning and budget development process? 3
How are areas for quality improvement in relation to clinical service delivery identified / agreed? 3
How are possible solutions to identified areas for quality improvement discussed? 3
How is implementation of solutions to identified areas for quality improvement supervised? 3
How is implementation of solutions to identified quality improvement needs monitored? 3
Is there a system at the facility to keep track of staff trainings and certifications? 3
Is there any mechanism to ensure the accuracy of recorded information on registries?
OBTAIN documents related to data quality checks (reports, feedback, checklists). 3
Number of sick children per month reported to HMIS 3
Source of supplies / consumables / equipment for this facility? 3
staff_status 3
What did the supervisor do during the last visit? 3
What is the procedure if a caregiver is unable to pay for any of the fees associated with healthcare provided to a sick child 0-59 months in this facility? 3
When was the last time a data quality check was undertaken? 3
Ampicillin 2
Are fees charged for the following services for healthcare provided to a sick child 0-59 months ? 2
Available modes of communication 2
Azithromycin 2
By what mechanisms are you informed by the referral facility? 2
Check completeness (e.g. blank counts) of (provisional or final) diagnoses over the last 20 entries 2
Check readability of (provisional or final) diagnoses over the last 20 entries 2
Check relevance of (provisional or final) diagnoses over the last 20 entries
(e.g. check diagnoses only, not symptoms are reported as diagnoses) 2
Clinical observations recorded in the registration or consultation ledger 2
Diazepam 2
Did the last supervision visit result in the planning of the following activities? 2
Do the funds or resources available to the facility depend on the facility’s past or current performance against specific metrics or goals (either overall or for specific designated targets or goals)? 2
Does the facility centrally record information about outgoing referrals? 2
For this health facility, is the budget based on categories of types of expenses (such as staffing, medicines, equipment etc.) or based on categories of the types of services (eg. HIV services, MCH services etc.), or both? 2
For what population is the facility policy to charge no fees? 2
Gentamicin 2
How many external supervision visits have occurred during the 12 last months? 2
How many healthcare providers receive supportive supervision? 2
Information received back on outgoing referrals from referral facility? 2
Laboratory records management system? 2
Laboratory register / ledger recording specific for children 0-59 months 2
Mechanisms used to directly inform the referral facility of the referral of a child 0-59 months? 2
Mechanisms used to inform caregivers about recommended follow-up of children 0-59 months 2
NG tubes 2
Pharmacy records management system? 2
Pharmacy register / ledger recording specific for children 0-59 months 2
Quinine (oral) 2
staff_time 2
staff_training 2
Stock outs in the last 7 days 2
Stock replenishment mechanism 2
Tasks for which computers are used? 2
Total number of children under 5 consultations reported to HMIS? 2
Total number of subgroup of # young infant/neonatal consultations reported to HMIS 2
Triage for children 0-59 months 2
When was the last time the facility committee happened? 2
When was the last time this facility received an external supervision visit? 2
Who is responsible for the day to day management of the facility? 2
Among those services, which ones are also provided to children 0-2 months? 1
Amoxicillin - dispersible tablets (500mg) 1
Amoxicillin - syrup / suspension (250 mg) 1
Artesunate (rectal) 1
At any time during the past 7 days has Haemocue been unavailable (for any reason)? 1
At any time during the past 7 days has malaria blood film microscopy been unavailable (for any reason)? 1
At any time during the past 7 days has malaria RDT been unavailable (for any reason)? 1
At any time during the past 7 days has pulse oximetry been unavailable in the outpatient department for any reason? 1
Chlorhexidine 1
Cloxacillin 1
cmpl_imci_training 1
Co-amoxiclav (Amoxicillin + Clavulanic acid) 1
Co-trimoxazole syrup / suspension 1
Communicable disease services 1
Consultation records management system? 1
Days of the week for which there are dedicated vaccination / growth monitoring services 1
Days per week facility for which there is normally at least one eligible healthcare professional is available to consult children 0-59 months? 1
Does someone else consult if the eligible provider is unavailable? 1
Does the facility centrally record outcomes of outgoing referrals? 1
Does this facility have budget earmarked for staff specifically hired for and designated to provide services for infants and/or children exclusively (and not adults)? 1
Does this facility receive any external supervision? 1
Does this health facility have any current goals or initiatives for quality improvement in relation to clinical service delivery? 1
Electricity availability in past 7 days (from main or backup source) during facility opening hours 1
For this facility, is there a budget or allocation of funds specifically designated to be used for services, staffing, medicines or equipment for infant and or child health curative services (and not other services such as PMTCT, HIV care and treatment or adult services)? 1
Frequency of facility committees 1
Gentian violet 1
Guidelines for Comprehensive Emergency Obstetric and Newborn Care (CEmONC) 1
Haemoglobin testing / estimation 1
Height board available 1
How frequently is data quality checked? 1
Is the emergency department separate from the outpatient area? 1
Malaria testing available 1
Mebendazole Tablet 1
Neonatal IV needles 1
Nitrofurantoin 1
Number of days for which there was no eligible healthcare provider available to consult children 0-59 months in the last 7 days? 1
Oral artemisinin combination therapy (paediatric dose) 1
Paediatric IV needles 1
Paracetamol syrup/suspension 1
path_imci_training 1
Personal protective equipment 1
Primaquine 1
Quinine (injection) 1
Registration records management system? 1
Rifampicin / Isoniazid / Pyrazinamide / Etambutol (RHZE) 1
Salbutamol 1
Segregation of waste 1
Services provided to children 2-59 months 1
Source of revenue or funding of this facility? 1
staff_childcare 1
Tests routinely perfomed 1
Toilet functional 1
Training log management system 1
Was IMCI part of the supervision visit? 1
What did the supervisor observe during the last visit? 1
What happens when the internet monthly data limit is reached? 1
What information does the facility centrally record during follow-up visits? 1
What quality improvement goals or initiatives in relation to clinical service delivery are currently ongoing in this facility? 1
Where activities were identified, were they followed upon as part of supervision? 1
Where do you refer children 0-59 months who require oxygen then? 1